Users of software and hardware devices demand high reliability and stability against device errors and failure. For example, integrity of data of a data processing device can be compromised when a device failure occurs. A test framework for a device can serve to detect errors, for example during development, and validate and verify quality attributes of the device. A test framework may for example be implemented on a computer system and may be used for facilitating the development, execution and documentation of a device test. It may for example comprise an integrated development environment (IDE) or integrated debugging environment that provides comprehensive facilities to computer programmers for software development.
A device test using a test framework can be carried out employing a test suite. For example, in software development, a test suite is a collection of test cases that are intended to be used to test a software program or software device to show that it has some specified set of behaviors. A test suite can contain detailed instructions or goals for a collection of test cases and information on the system configuration to be used during testing. A group of test cases may also contain prerequisite states or steps, and descriptions of the following tests.
A device to be tested may often be logically or physically divided into subcomponents that can be tested hierarchically. The execution of a test case is usually organized in a sequence of test execution steps. Typically, an error is blocking the next test execution steps, limiting automatic test coverage of the device. After eliminating the error, the test may be started again. However, in large test suites, for example comprising interconnected test cases, error propagation can occur, requiring time-consuming manual investigation of errors.